WO 99/33265 



Page 1 of 24 



PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 
H04N 5/91 



(21) International Application Number: PCT/US98/22599 

(22) International Filing Date: 23 October 1998 (23.10.98) 



23 December 1997 (23.12.97) US 



(71) Applicant (for all designated States except US): INTEL COR- 
PORATION [US/US]; 2200 Mission College Boulevard, 
Santa Clara, CA 95052 (US). 

(72) Inventor; and 

(75) Inventor/Applicant (for US only): O'CONNOR, Dennis, M. 
[US/US]; 12130 E. Via De Palmas, Chandler, AZ 85249 
(US). 

(74) Agents: TAYLOR, Edwin, H. et al.; Blakely, Sokoloff, Taylor 
& Zafman LLP, 7th floor, 12400 Wilshire Boulevard, Los 
Angeles, CA 90025 (US). 



(11) International Publication Number: WO 99/33265 

(43) International Publication Date: 1 July 1999 (01 .07.99) 



es: AL, AM, AT, AT (Utility model), AU 
(Petty patent), AZ, BA, BB, BG, BR, BY, CA, CH, CN, CU, 
CZ, CZ (Utility model), DE, DE (Utility model), DK, DK 
(Utility model), EE, EE (Utility model), ES, EL FI (Utility 
model), GB, GD, GE, GH, GM, HR, HU, ID, IL, IS, JP, 
KE, KG, KP, KR, KZ, LC, LK, LR, LS, LT, LU, LV, MD, 
MG, MK, MN, MW, MX, NO, NZ, PL, PT, RO, RU, SD, 
SE, SG, SI, SK, SK (Utility model), SL, TJ, TM, TR, TT, 
UA, UG, US, UZ, VN, YU, ZW, ARJPO patent (GH, GM, 
KE, LS, MW, SD, SZ, UG, ZW), Eurasian patent (AM, AZ, 
BY, KG, KZ, MD, RU, TJ, TM), European patent (AT, BE, 
CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, IT, LU, MC, 
NL, PT, SE), OAPI patent (BF, BJ, CF, CG, CI, CM, GA, 
GN, GW, ML, MR, NE, SN, TD, TG). 



(54) Title: METHOD OF TIME SHIFTING TO SIMULTANEOUSLY RECORD AND PLAY A DATA STREAM 



(57) Abstract 



s received at 
The video stream 
t (108). 



A video s 
input (102). 
recorded to a 
While the video stream continues tc 
recorded, a portion of the video stream 
retrieved from the storage unit (108) 
d provided at an output (120). In one 
embodiment, the storage medium (108) 
random access storage unit, such as 
a hard disk. 




WO 99/33265 



Pag e 2 of 24 



Codes used to identify States 



FOR THE PURPOSES OF INFORMATION ONLY 
party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



Australia 

Bosnia and Herzegovina 

Barbados 

Belgium 

Burkina Faso 

Bulgaria 



Switzerland 
C8te d'lvoire 
Cameroon 

Cuoa 

Czech Republic 
Germany 

Estonia 



Senegal 
Swaziland 



Hungary 
Ireland 



Kyrgyzstan 



Liechtenstein 
Sri Lanka 



Trinidad and Tobago 
United States of America 



WO 99/33265 PCTAJS98/22599 
METHOD OF TIME SHIFTING TO SIMULTANEOUSLY RECORD AND 
PLAY A DATA STREAM 



BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The described invention relates to the field of imaging. In particular, 
the invention relates to the recording and playing back of a video stream. A 
video stream includes any combination of audio and/ or video data streams. 
However, the concepts disclosed may be applied to other types of data 
streams. 

2. Description of Related Art 

Video streams have typically been recorded on analog media such as a 
video cassette. A video cassette recorder (VCR) is used to record the video 
stream on the video cassette. The video stream may come via a broadcast 
signal, via cable, via satellite signal, or from another video playback device. 
Once the video stream has been recorded, the VCR is used to rewind the 
recording medium and play what was recorded. However, due to the 
nature of the analog medium, once the VCR has started recording, it is not 
possible to play back the portion of the video stream that has already been 
recorded until the recording session is terminated. 

For example, imagine that a person sets up a VCR for recording a one 
hour show because he knows that he will miss the first 15 minutes of the 
show. When he arrives home 15 minutes into the show, he will have to 
wait for the entire show to be recorded before he can start watching the 
program from the beginning. A way of being able to view the show from 
the beginning without having to wait for the recording session to terminate 
is desirable. 

SUMMARY OF THE INVENTION 

A method of providing a time-shifted video stream is disclosed. A 
video stream is received at an input. The video stream is recorded to a 
storage unit. While the video stream continues to be recorded, a portion of 
the video stream is retrieved from the storage unit and provided at an 
output. In one embodiment, the storage unit is a random access storage 
unit, such as a hard disk. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows a block diagram of a video record and playback system 
in accordance with one embodiment of the invention. 

Figure 2 shows a flowchart of the method of providing a time-shifted 
video stream. 

Figure 3 shows one embodiment of performing block 206 of Figure 2. 

Figure 4 shows one embodiment of an apparatus for storing the 
video stream on a hard disk. 

Figure 5 shows an exemplary method of using the storage unit as a 
temporary buffer. 

Figure 6 shows a flowchart of the playback of a video stream catching 
up to the incoming video stream. 

Figure 7 shows another embodiment, in which the user is able to 
suspend the display of the incoming video stream. 

Figure 8 shows an example of a display screen with several image 
frames taken at different times. 

DETAILED DESCRIPTION 
The Apparatus 

Figure 1 shows a block diagram of a video record and playback system 
100 in accordance with one embodiment of the invention. A video stream 
is received at the VIDEO IN 102. The video stream may be provided by a 
camera, a television signal, broadcast, cable, or satellite signals, or another 
video playback device. In one embodiment, the VIDEO IN 102 performs an 
A to D function that takes analog NTSC signals coming in and decodes the 
video stream into a digital video bit stream. In a different embodiment, the 
video is already in digital form. In one embodiment, video record and 
playback system 100 is a personal computer system, and the VIDEO IN 102 is 
a video capture card plugged into the personal computer system. 

The video stream output of the VIDEO IN 102 is optionally 
compressed at compression unit 104. In one embodiment, the video is 
already compressed, such as would be the case with an MPEG2 video signal, 
and no further compression is needed. The video stream is then stored into 
storage unit 108. Buffer unit 106 may be used as a temporary storage unit for 
providing larger sequential blocks of video data to the storage unit 108. 

In one embodiment, storage unit 108 is a random access memory unit 
that allows relatively quick access to any portion of the stored video stream. 
A hard disk is an example of a random access memory unit. A hard disk 
-2- 
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can typically access stored data with access times that are on the order of 10 
ms. In contrast, video cassette tapes are mechanically wound to get to a 
particular point in the video stream. 

The video stream is played back by reading the video stream from the 
storage unit 108. If the video stream was compressed in compression unit 
104, then a decompression unit 110 decompresses the retrieved video 
stream. The video stream is provided to a VIDEO OUT port 120, to which a 
monitor or other display device is connected to provide sound and /or 
video to a user. 

A removable storage unit 122 may also be included in video record 
and playback system 100. Examples of removable storage units are a 
writeable CD ROM, writeable DVD, a flash memory, or another hard disk. 
The removable storage unit 122 allows a user to transfer a recording of a 
video stream stored in storage unit 108 to the removable storage unit 122. 

In one embodiment, a processor 130 controls the operations of the 
video record and playback system 100. The compression unit 104 and 
decompression unit 110 may be implemented in hardware, or the 
compression and decompression functions of units 104 and 110 may be 
performed by the processor 130. Processor 130 receives instructions from 
firmware/memory 140, using technology that is well-known. 

Record and Playback Cycle 

Figure 2 shows a flowchart of the method of providing a time-shifted 
video stream. The flowchart begins at block 200, and continues at block 202 
at which the video stream is received at an input. The video stream begins 
being recorded at block 204. At block 206, playback of the video stream that 
has been recorded is performed by retrieving a portion of the video stream 
from the random access storage unit while the recording of the incoming 
video stream continues. The retrieved portion of the video stream is time- 
shifted from the incoming video stream by a time delay. 

At block 208, the portion of the video stream retrieved from the 
random access storage unit is provided at an output of the video record and 
playback system 100. The retrieved portion of the video stream can then be 
displayed by a television or other display device. 

By using the method of Figure 2, the record and playback functions 
are decoupled from one another. The user may now begin watching a 
recorded TV show from the beginning, i. e., prior to the show being 
completely recorded. 
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Figure 3 shows one method for performing block 206 of Figure 2 of 
retrieving a portion of the video stream from the random access storage 
unit while continuing to record the incoming video stream. In the 
disclosed embodiment, the simultaneous recording and playback of the 
video stream is performed by alternately performing, or multiplexing, one 
or more stores of the video stream to the random access storage unit with 
one or more reads of the video stream from the random access storage unit. 
The multiplexed stores and reads occur quickly enough that the user does 
not notice an appreciable delay in the playback of the video stream, and 
none of the incoming video stream is lost, i.e., all of the video stream is 
recorded. Thus, the record and playback are simultaneous from the user's 
point of view. 

In one embodiment, the random access storage unit is a hard disk. 
The retrieval of the time-shifted video signal from the hard disk is 
performed at a first physical location (or sector) of the hard disk, and the 
storing to the hard disk of the incoming video stream is performed at a 
different physical location (or sector) on the hard disk. Because it takes 
more time to jump back and forth between different sectors of the hard disk 
than the time it takes to read and write to sequential locations in the same 
sector, data may be buffered to minimize the number of accesses to and 
from the hard disk using buffer 106. This increases the amount of data 
transferred per access. 

Additionally, because of time constraints for reading and writing to 
the hard disk, data may be compressed and decompressed to speed transfers 
to and from the hard disk. 

In block 302, the video stream is stored to a random access storage 
unit. The video stream is optionally compressed and /or buffered prior to 
storage. In block 304, the video stream retrieved from the random access 
storage unit. Buffering and/or decompression may be performed before 
providing the retrieved video stream to the VIDEO OUT port 120. 

From block 304, the flowchart proceeds at block 306 which stores the 
next portion of the video stream in a manner similar to that described in 
block 302. At block 308, the next portion of the video stream is retrieved in 
a manner similar to that described in block 304. This process is repeated 
until either the recording or playback cycle is terminated. 

Figure 4 shows one embodiment of an apparatus for storing the 
video stream in the storage unit 108. In this embodiment, the video stream 
is stored as separate files 001 to 009 on a hard disk, for example. The 
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processor 130 keeps track of the file and offset into the file of the data being 
played back, as well as the file and offset into the file of the data being 
recorded. If the random access storage unit is fast enough, the more than 
one video stream can be recorded and played back at the same time. 

Temporary buffering (Circular buffering) 

Due to the nature of the random access storage unit being capable of 
easily recording over itself, the random access storage unit may act as a 
temporary buffer for recording the latest portion, or X number of minutes, 
of an incoming video stream, where X is set up based upon the size of the 
storage unit. In one embodiment, X could be set up to be the entire storage 
unit. As newer portions of the video stream are received, they will 
overwrite the older portions of the video stream saved in the random 
access storage unit. In this manner, the temporary buffering of the video 
stream acts as a circular buffer. In one embodiment, the processor 130 
maintains pointers to the beginning and ending points of the temporary 
buffer. The processor 130 reassigns the pointers as newer portions of the 
video stream are received and/or older portions of the video stream are 
overwritten. 

Figure 5 shows a flowchart of one method for using the storage unit 
as a temporary buffer. The flowchart starts at block 500 from which it 
continues at block 502, at which the video stream is received at an input. 
Recording of the video stream to the storage unit begins at block 504. At 
block 506, older portions of the video stream are deleted as newer portions 
of the video stream are stored to the storage unit. 

A user may initiate a playback cycle following block 506. For example, 
this may occur when the user wishes to re-view a video clip that he just 
saw. In one embodiment, the user stops recording to the temporary buffer 
and plays back the last portion of the temporary buffer. 

However, it may be more desirable to the user to be able to continue 
recording as shown at block 508. A record and playback cycle (as described 
with respect to Figure 2) is started, in which the incoming video stream is 
recorded while the user re-views the last portion of the temporary buffer. 
In this manner, after re-viewing the desired video clip, the user can resume 
sequentially watching the video stream from the point of the video clip. 

At block 510, after the record and playback cycle is completed, all or 
part of the temporary buffer may be saved. Since the temporary buffer store 
the latest X minutes of the video stream prior to the record and playback 
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cycle, all or part of the temporary buffer may be allocated to the portion of 
the video stream saved during the record and playback cycle. Other 
portions of the video stream may then be deleted from the storage unit, or 
they may be marked as over-writeable and used as a new temporary buffer. 

Playback catching up to Incoming Video Stream 

Figure 6 shows a flowchart of a method for playing back of a video 
stream in a manner that allows it to catch up to the incoming video stream. 
Usually, during simultaneous playback and recording of the same video 
stream, the playback of the video stream is time-shifted from the incoming 
video stream by a time delay. However, if the playback is performed at an 
overall rate faster than the rate at which the incoming video stream is 
received, then the playback will catch up to the incoming video stream. 

For example, playback of the video stream will have an overall rate 
faster than the rate of the incoming video stream if the playback is fast 
forwarded, or if segments of the playback are skipped altogether. When the 
time delay of the time-shifted video stream being played back falls below a 
certain threshold, the video and playback system 100 will cease providing 
the time-shifted video stream from the storage unit. Instead, the incoming 
video stream will be provided to the VIDEO OUT port 120 directly. In one 
embodiment, a bypass 142, as shown in Figure 1, allows the incoming video 
stream to be provided to the VIDEO OUT port 120 directly. 

When this happens, the user has caught up to the "live" broadcast, 
i.e., the incoming video stream. The user may terminate the recording 
cycle, if he wishes. Alternatively, the user can put the video record and 
playback system 100 back into the temporary buffering mode in which only 
the latest portion of the video stream is stored. 

"Suspending" Incoming Video Stream 

Figure 7 shows another embodiment, in which the user is able to 
suspend the display of the incoming video stream. This can be used for 
example, when the user is interrupted, and wishes to continue viewing the 
video stream after the interruption. When interrupted, the user signals to 
the video record and playback system 100 to suspend the display of 
incoming video stream. This can be done via a remote control, for 
example. At block 702, the video out is suspended. In one embodiment, the 
VIDEO OUT continues to provide a still image of the image present at the 
VIDEO OUT when the suspend was encountered. 
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At block 704 of the flowchart, the incoming video stream is recorded 
but is not displayed to the monitor. Instead the playback is paused at the 
point at which the user indicated that the incoming video stream be 
suspended. When the user is ready to view the video stream again, he can 
signal the video record and playback system 100 to un-suspend the video 
stream so that it plays back from the point at which the video stream was 
suspended, as shown in block 706. 

The user may then view the video stream time shifted by the 
amount of time that he suspended the incoming video stream, or he may 
fast forward (or rewind) through the time-shifted video stream. When 
playback of the time-shifted video stream catches up to the point at which 
the incoming video stream is being recorded, the record and playback 
system 100 may display the incoming video stream directly from incoming 
video stream without retrieving the video stream from the storage unit, as 
described with respect to Figure 6. The recording of the video stream may 
then be terminated by the user, if desired. 

Story Boarding: Fast Forward and Rewind 

Employing a random access storage unit for storage of the video 
stream facilitates jumping to various points within the video stream. One 
way of jumping is by retrieving different time slices of the video stream. 
For example, an image frame from the video stream can be retrieved from 
the storage unit at 1 minute intervals from a current position of the video 
stream. In one embodiment, an image frame at the current playback 
position + 1 minute, current playback position + 2 minutes, and so forth are 
retrieved and displayed on the TV screen. 

Figure 8 shows an example of a display screen 800 which displays 
several image frames taken from the video stream at different times. In 
Figure 8, the current playback position is designated as X. In one 
embodiment, the time interval, T, is user programmable. The intervals 
may be multiples of the time interval, as shown. A small interval may be 
used if the user wishes to skip a commercial, which usually lasts only a few 
minutes. Longer intervals such as a half hour may be useful for 
determining which movies are recorded on a storage unit. 

After the image frames are displayed, the user is able to select one of 
the frames as a new starting point to which to begin an operation, such as a 
playback or record operation. By using such a story boarding method, it is 
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easy for a user to quickly jump to a desired location within the video 
stream. 

In one embodiment, the record and playback system 100 is able to detect a 
black screen or fade out, such as those which accompany the beginning or 
end of a commercial. This is useful in editing a video stream. 

Editing 

Having the video stream stored on a random access storage unit such 
as a hard disk allows for easy editing of the video stream. Individual frames 
of the video stream may be deleted or replaced. For example, a commercial 
may be replaced by a fade-to-black sequence. 

Once the video stream on the storage unit has been edited, it can be 
stored to a more permanent medium such as a writeable CD-ROM, flash 
memory, or another hard disk via the removable storage unit 122. 

Conclusion 

Thus, a method of simultaneously recording and playing back a time- 
shifted video stream is disclosed. The specific arrangements and methods 
described herein are merely illustrative of the principles of this invention. 
For example, the same method may be used to store and retrieve other types 
of data streams besides video streams. Numerous modifications in form 
and detail may be made without departing from the scope of the described 
invention. Although this invention has been shown in relation to a 
particular embodiment, it should not be considered so limited. Rather, the 
described invention is limited only by the scope of the appended claims. 
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1. A method of displaying a video stream comprising the steps of: 

(a) receiving a video stream at an input; 

(b) recording the video stream to a storage unit; 

(c) retrieving a portion of the video stream from the storage unit while 
continuing to record the video stream in the step (b); and 

(d) providing at an output the portion of the video stream retrieved in the 
step (c). 

2. The method of claim 1 wherein the step (b) is further comprised of 
recording the video stream to a hard disk. 

3. The method of claim 1 wherein the step (b) is further comprised of 
recording the video stream to a random access media. 

4. The method of claim 1 wherein the step (c) of retrieving the portion of 
the video stream retrieves the video stream recorded in the step (b) time 
shifted by a time delay, and wherein when the time delay falls below a 
predetermined threshold, the step (c) of retrieving a portion of the video 
stream from the storage unit is discontinued, and the step (d) of providing 
at an output the portion of the video stream retrieved in the step (c) is 
substituted by the step of providing at an output the video stream as it is 
received in the step (a). 

5. The method of claim 1 further comprising the step of: 

(e) retrieving two or more frames of the video stream shifted by different 
time delays; 

(f) displaying the two or more frames of the video stream; and 

(g) allowing a user to select one of the frames of the video stream as a 
starting point for playing back the video stream. 

6. A method of displaying a video stream comprising the steps of: 

(a) receiving the video stream at an input; 

(b) initiating one or more stores of the video stream into a storage unit; 
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(c) initiating one or more reads of the video stream from the storage unit, 
wherein the one or more stores of the step (b) are multiplexed with the one 
or more reads of the step (c); and 

(d) displaying the video stream. 

7. The method of claim 6 wherein the one or more reads of the video 
stream from the storage unit are used to play back the video stream at a rate 
faster than it is being stored in the step (b). 

8. The method of claim 6 wherein the one or more reads of the step (c) 
correspond to the video stream of the step (a) shifted by a time delay, 
wherein when the time delay falls below a predetermined amount of time, 
the one or more reads from the storage unit cease, and the step (d) is further 
comprised of the step of: 

(d) displaying the video stream received in the step (a). 

9. The method of claim 6 wherein the one or more reads of the step (c) 
access the video stream offset by a time delay from the video stream being 
stored in the step (b), the time delay being variable over time. 

10. The method of claim 6 further comprising the step of: 

(e) retrieving two or more frames of the video stream shifted by different 
time delays; 

(f) displaying the two or more frames of the video stream; and 

(g) allowing a user to select one of the frames of the video stream as a 
starting point for playing back the video stream. 

11. A method of displaying a video stream, the method comprising the 
steps of: 

(a) receiving the video stream; 

(b) storing the video stream received in the step (a) to a storage unit; and 

(c) displaying the video stream at least initially delayed by a time delay, 
wherein when the time delay is greater than a predetenruned threshold the 
video stream is displayed from the storage unit, and wherein when the 
time delay is less than the predetermined threshold the video stream is 
displayed from the video stream received in the step (a). 
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12. The method of claim 11, wherein the step (b) of storing the video stream 
to the storage unit comprises performing one or more stores to the storage 
unit, and the step (c) of retrieving the video stream from the storage unit 
when the time delay is greater than the predetermined threshold comprises 
performing one or more reads from the storage unit, and the one or more 
stores to the storage unit are multiplexed with the one or more reads from 
the storage unit. 

13. The method of claim 12, wherein the storage unit is a random access 
memory unit, and the one or more reads from the storage unit retrieve the 
video stream with one or more time delays that are user-specified. 

14. The method of claim 11, wherein the step (b) of storing the video stream 
to the storage unit further comprises compressing the video stream prior to 
storing the video stream to the storage unit. 

15. The method of claim 11, wherein the step (c) of retrieving the video 
stream from the storage unit when the time delay is greater than the 
predetermined threshold comprises decompressing the video stream after 
retrieving the video stream from the storage unit. 

16. A method of recording a video stream, the method comprising the steps 
of: 

(a) receiving the video stream; 

(b) initiating storing the video stream in a storage unit; 

(c) deleting older portions of the video stream from the storage unit as new 
portions of the video stream are stored to the storage unit; 

(d) receiving a user input designating a start of a record cycle; and 

(e) storing the video stream received after the start of the record cycle. 

17. The method of claim 16 further comprising the steps of: 

(f) maintaining at least a portion of the video stream prior to the record 
cycle. 

18. A method of recording a video stream, the method comprising the steps 
of: 

(a) receiving the video stream; 

(b) displaying the video stream as it is received; 
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(c) responsive to a first user input, suspending display of the video stream 
and storing to a storage medium the video stream received subsequent to 
the first user input. 

19. The method of claim 18 further comprising: 

(d) responsive to a second user input, reading the video stream from the 
storage medium, and providing the video stream at an output while 
continuing to record the video stream as it is received. 

20. The method of claim 19 wherein reading the video stream from the 
storage medium in the step (d) is performed at an overall rate faster than a 
rate that the video stream is being received. 

21. The method of claim 20 wherein reading the video stream from the 
storage medium is performed by skipping portions of the video stream. 

22. The method of claim 19 wherein the step (d) is further comprised of: 

(d) reading the video stream from the storage medium to provide the video 
stream at the output. 

23. The method of claim 22 wherein the video stream read in the step (d) 
corresponds to the video stream received in the step (a) shifted by a time 
delay, wherein when the time delay falls below a predetermined amount of 
time, the method further comprising the step of: 

(e) providing at an output the video stream received in the step (a) without 
first storing the video stream to the storage medium. 

24. The method of claim 19 further comprising the step of: 

(e) retrieving two or more frames of the video stream shifted by different 
time delays; 

(f) displaying the two or more frames of the video stream; and 

(g) allowing a user to select one of the frames of the video stream as a 
starting point for playing back the video stream. 

25. A method of recording a video stream, the method comprising the steps 
of: 

(a) receiving the video stream; 
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(b) storing the video stream into a temporary buffer, wherein older portions 
of the video stream are continually updated by new portions of the video 
stream; and 

(c) responsive to a user input, displaying a portion of the video stream 
stored in the temporary buffer. 
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FIG. 1 
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FIG. 2 
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FIG. 3 



COMPRESS/BUFFER VIDEO 
STREAM 



STORE VIDEO STREAM INTO THE 
RANDOM ACCESS STORAGE UNIT 



COMPRESS/BUFFER VIDEO 
STREAM 



STORE VIDEO STREAM INTO THE 
RANDOM ACCESS STORAGE UNIT 



RETRIEVE VIDEO STREAM FROM 
THE RANDOM ACCESS STORAGE 
UNIT 

BUFFER/DECOMPRESS VIDEO 
STREAM 



RETRIEVE VIDEO STREAM FROM 
THE RANDOM ACCESS STORAGE 
UNIT 

BUFFER/DECOMPRESS VIDEO 
STREAM 



* r ~ 



W O 99 /33265 



Page 19 of 24 



FIG. 4 



PCT/US98/22599 



PLAYBACK 
FILE: 003 
OFFSET: OFF_1 



RECORD 
FILE: 009 
OFFSET: OFF_2 



PROCESSOR 



WO 99/33285 



Page 20 of 24 



PCT/US98/22599 



5/8 



FIG. 5 
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